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IMAGE CALIBRATION METHOD, IMAGE CALIBRATION PROCESSING DEVICE, 
AND IMAGE CALIBRATION PROCESSING TERMINAL 

Technical Field 

5 The present invention relates to an image calibration method 
of performing a camera calibration on an image photographed by 
a general camera to correct distortions due to the lens in the 
image photographed by the camera, and in particular to an image 
calibration method which does not impose precise calibration 

10 calculations on the side of a camera user. 

The present invention also relates to an image calibration 
processing device capable of performing such precise 
calibration calculations as to allow, for example, stereo 
viewing- The present invention further relates to an image 

15 calibration processing terminal capable of transmitting an 
image photographed by a camera or photographing information on 
the camera to an image calibration processing device, and 
displaying a corrected subject image on which a calibration 
calculation has been performed. 

20 

Background Art 

Conventionally, when a plurality of single photographs are 
photographed and a panorama photograph is formed therefrom, it 
is necessary to correct the photographs to be combined so as 
25 to remove image distortions. Also, when an image photographed 
two-dimensionally is used to measure dimensions of an object, 
it is necessary to correct the photographed image so as to remove 
image distortions. 

When a three-dimensional shape of a measuring object is 
30 measured based on two images photographed, measurement points 
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on the measuring object are found based on the principle of a 
stereo method. In order to perform a three-dimensional 
measurement, a calibration of a camera, namely interior 
localizations (amount of lens distortions, focal length, and 
5 principal point position) of the camera, needs to be performed 
beforehand. Since the three-dimensional shape of the 
measuring object is found using the parallax between the images 
photographed by the camera, the distortions due to the camera 
lens used in the photographing need to be corrected very 
10 precisely. 

[Patent Document 1] 

JP-A-Hei09-329418 (Paragraphs 0006 and 0020, and FIG. 4 and 
FIG. 5) 

[Patent Document 2] 
15 JP-A-Heill-351865 (Paragraphs 0002 and 0004, and FIG. 1 and 

FIG. 3) 

Disclosure of the Invention 

Problem to be Solved by the Invention 

20 However, performing a calibration of a camera as described 
above requires three-dimensional targets distributed over a 
wide range and measured accurately, an analysis software 
program, know-how for photographing, and the like. Therefore, 
the calibration work would be too complicated work for 

25 construction site or archaeological excavation site workers, 
clothing store clerks, or the like, for example. A calibration 
object suitable for calibration of a camera was disclosed by 
the present applicant in Patent Document 1 mentioned above. 
Even when such a calibration object is used, a software program 

30 for calibration needs to be purchased and the know-how on the 
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photographing object, photographing method, and analysis is 
required. Therefore, formation of a panorama photograph by 
combining a plurality of single photographs, measurement of 
dimensions of an object using an image photographed two- 
5 dimensionally, or three-dimensional measurement based on the 
principle of a stereo method would evoke a great resistance to 
construction site or archaeological excavation site workers, 
clothing store clerks, or the like who are unskilled in those 
works . 

10 The present invention has been made in view of the foregoing 
problems, and has a first object of providing an image 
calibration method that allows acquisition of a corrected 
subject image in which distortions due to the camera lens are 
corrected, even for those unskilled in three-dimensional 

15 measurement based on the principle of a stereo method, precise 
two-dimensional measurement , and precise image formation, such 
as construction site or archaeological excavation site workers, 
clothing store clerks, or the like. 

A second object of the present invention is to provide an image 

20 calibration processing device that allows acquisition of a 
corrected subject image in which distortions due to the camera 
lens are corrected to such a degree as to allow three- 
dimensional measurement, precise two-dimensional measurement, 
and precise image formation in a concise manner. A third object 

25 of the present invention is to provide an image calibration 
processing terminal that allows access to an image calibration 
processing device and acquisition of a corrected subject image 
in which distortions due to the camera lens are corrected in 
a concise manner. 
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Means for Solving the Problem 

In order to achieve the first object , an image calibration 
method according to the present invention comprises, as shown 
for example in FIG. 1 and FIG. 2, the steps of: receiving, at 
5 an image calibration processing device 10 , a calibration 
request from a user terminal 20 (S10) ; receiving, at the image 
calibration processing device 10, a calibration object image 
photographed in accordance with an instruction for calibration 
photographing (S16) ; performing a calibration process using the 

10 received calibration object image (S26) ; receiving, . at the 
image calibration processing device 10, a subject image 
transmitted from the user terminal 20 and photographed by a 
camera 2 used to photograph the calibration object image (S30) ; 
performing a calibration on the subject image in accordance with 

15 results of the calibration process to form a corrected subject 
image (S32); and returning the formed corrected subject image 
from the image calibration processing device 10 to the user 
terminal 20. 

According to an image calibration method including steps as 
20 described above, the image calibration processing device 10 
receives a calibration object image photographed by the camera 
2 as an object of calibration, performs a calibration process 
on the calibration object image received, and calculates the 
results of the calibration process necessary for correction of 
25 the lens distortions or the like. Then, the image calibration 
processing device 10 receives a subject image photographed by 
the camera 2, performs a calibration on the subject image in 
accordance with the results of the calibration process, and 
forms a corrected subject image. The corrected subject image 
30 is returned from the image calibration processing device 10 to 
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the user terminal 20 . In this manner, the user terminal 20 needs 
only to transmit a calibration object image photographed by the 
camera 2, and then a subject image photographed by the camera 
2, to the image calibration processing device 10 in accordance 
5 with an instruction for calibration photographing. Meanwhile, 
the image calibration processing device 10 bears the burden of 
complicated calculation processing for a corrected subject 
image, thereby reducing the calculation load for the user 
terminal 20. Since the user terminal 20 is used at a work site 
10 or a travel destination, a light calculation load will reduce 
the power consumption and stabilize the workability of the user 
terminal 20. 

. Preferably, in the image calibration method as described above, 
as shown for example in FIG. 18 and FIG. 19, the calibration 

15 object image and the subject image are a common image in which 
a calibration object 4 and a subject 6 are photographed at a 
time, and the method further comprises the step of performing 
a three-dimensional measurement on the subject using the 
corrected subject image. Then, a three-dimensional image 

20 measurement on the subject can be performed. Thus, the image 
calibration processing device 10 can provide a three- 
dimensional image measurement to users who require it, as an 
additional service to the calibration process. 

In order to achieve the first object, an image calibration 

25 method according to the present invention comprises, as shown 
for example in FIG. 14 and FIG. 15, the steps of: receiving, 
at an image calibration processing device 30, a calibration 
request from a user terminal 4 0 (S100) ; receiving, at the image 
calibration processing device 30, a calibration object image 

30 photographed in accordance with an instruction for calibration 



6 



photographing (S106) ; performing a calibration process using 
the received calibration object image (S116) ; and transmitting 
results of the calibration process calculated in the 
calibration process from the image calibration processing 
5 device 30. And, the user terminal 40 performs a calibration 
on a subject image photographed by a camera used to photograph 
the calibration object image (S120), using the results of the 
calibration process transmitted from the image calibration 
processing device 30 , to form a corrected subject image (S122) - 

10 According to an image calibration method including steps as 
described above, the image calibration processing device 30 
receives a calibration object image photographed by the camera 
2 as an object of calibration, performs a calibration process 
on the calibration object image received, and calculates the 

15 results of the calibration process necessary for correction of 
the lens distortions or the like. The calculated results are 
returned to the user terminal 40. Meanwhile, the user terminal 
4 0 performs a calibration on a subject image photographed by 
the camera 2 using the results of the calibration process 

20 transmitted from the image calibration processing device 30, 
and forms a corrected subject image. Then, the user terminal 
4 0 can obtain a corrected subject image from the subject image 
photographed by the camera 2, using the results of the 
calibration process obtained from the image calibration 

25 processing device 30. At this time, since the user terminal 
40 bears the burden of calculation processing for a corrected 
subject image, a corrected subject image can be reliably 
obtained even in cases where a communication line between the 
user terminal 4 0 and the image calibration processing device 

30 30 has a narrow bandwidth or is not stable. 
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Preferably, in the image calibration method as described above, 
as shown for example in FIG. 20 and FIG. 21, the calibration 
object image and the subject image are a common image in which 
a calibration object 4 and a subject 6 are photographed at a 
5 time, and the user terminal 40 is configured further to perform 
a three-dimensional measurement on the subject using the 
corrected subject image (S234) . 

Preferably, in the image calibration method as described above, 
the method further includes the step of transmitting the 

10 instruction for calibration photographing from the image 
calibration processing device (10, 30) to the user terminal (20, 
40) (S12, S102) . There may be cases where an instruction for 
calibration photographing is not carried to a site or a travel 
destination by mistake. In such cases, an instruction can be 

15 downloaded to the user terminal (20, 40) by making an access 
to the image calibration processing device (10, 30) . 
Preferably, the instruction includes an image showing a 
photographing state. Such an image will facilitate 

understanding of photographing work for a calibration object 

20 image, for an operator who photographs a calibration object 
image by the camera 2 as an object of calibration. 

Preferably, the image calibration method as described above 
includes the step of: receiving at the image calibration 
processing device (10, 30) at least a focal length in the 

25 photographing and a model of the camera, or a focal length in 
the photographing and a size of an imaging device used in the 
photographing, together with the calibration object image 
transmitted from the user terminal (20, 40) (S16, S106) . The 
focal length in the photographing and the model of camera, or 

30 the focal length in the photographing and the size of an imaging 
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device used in the photographing, are useful information for 
calibration processing. 

Preferably, the image calibration method as described above 
includes the step of: reading, at the image calibration 
5 processing device (10, 30), data necessary for the calibration 
process from photographing information affixed to the 
calibration object image transmitted from the user terminal (20, 
40) (S16, S106) . The read data is used for the calibration 
process'. 

10 Preferably, in the image calibration method as described above, 
the results of the calibration process includes at least one 
of a principal point position, distortion data, and a focal 
length of a lens used in the photographing, of the camera 
involved in the photographing of the calibration object image. 

15 Then, parameters necessary for calculation processing for a 
corrected subject image can be obtained. 

Preferably, the image calibration method as described above 
includes the step of: accumulating the calculated results of 
the calibration process in a calibration database (19, 39) when 

20 the image calibration processing device (10, 30) performs the 
calibration process using the received calibration object image. 
The results of the calibration process accumulated may be 
utilized, for example, in CRM (Customer Relation Management) 
or the like. 

25 In order to achieve the first object, an image calibration 
method according to the present invention is, as shown for 
example in FIG. 12 and FIG. 13, used with an image calibration 
processing device 10 including a calibration database 19 in 
which results of a calibration process are accumulated 

30 according to a model of a camera. That is, the method comprises 
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the steps of: receiving, at the image calibration processing 
device 10, a calibration request from a user terminal 20, 
together with a model of a camera 2 used to photograph the subject 
image (S10, Sllb) ; retrieving, from the calibration database 
5 19, results of the calibration process corresponding to the 
received model of the camera; performing a calibration on the 
calibration object image using the retrieved results of the 
calibration process to form a corrected subject image (S32); 
and returning the formed corrected subject image from the image 

10 calibration processing device 10 to the user terminal 20. 

According to an image calibration method including steps as 
described above, the results of the calibration process are 
accumulated beforehand in the calibration database 19 according 
to the model of the camera 2 as an object of calibration. Since 

15 the image calibration processing device 10 receives a 
calibration request from the user terminal 20, together with 
the model of the camera 2 used to photograph the subject image, 
results of the calibration process accumulated in the 
calibration database 19 and corresponding to the model of the 

20 camera 2 can be read. In cases where the model of the camera 
2 and the results of the calibration process correspond to each 
other within a certain tolerable range, the results of the 
calibration process accumulated in the calibration database 19 
and corresponding to the model of the camera 2 can be used and 

25 the calibration object image photographed in accordance with 
an instruction for calibration photographing does not need to 
be used. Thus, the user terminal 20 does not need to transmit 
the calibration object image photographed by the camera 2 to 
the image calibration processing device 10, and the image 

30 calibration processing device 10 does not need to calculate the 
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results of the calibration process necessary for correction of 
the lens distortions or the like, thereby allowing the work to 
go smoothly. 

In order to achieve the first object, an image calibration 
5 method according to the present invention is, as shown for 
example in FIG. 16 and FIG. 17, used with an image calibration 
processing device 30 including a calibration database 39 in 
which results of a calibration process are accumulated 
according to a model of a camera. That is, the method comprises 

10 the steps of: receiving, at the image calibration processing 
device 30, a calibration request from a user terminal 40, 
together with a model of a camera 2 used to photograph a subject 
image (S100, SlOlb) ; retrieving, from the calibration database 
39, results of the calibration process corresponding to the 

15 received model of the camera (SlOlc) ; and transmitting the 
retrieved results of the calibration process from the image 
calibration processing device 30 (S118) . And, the user 
terminal 40 performs a calibration on the subject image using 
the results of the calibration process transmitted from the 

20 image calibration processing device 30, to form a corrected 
subject image (S122) . 

According to an image calibration method including steps as 
described above, the results of the calibration process are 
accumulated beforehand in the calibration database 39 according 

25 . to the model of the camera 2 as an object of calibration. Thus, 
in cases where the model of the camera 2 and the results of the 
calibration process correspond to each other within a certain 
tolerable range, the results of the calibration process 
accumulated in the calibration database 39 and corresponding 

30 to the model of the camera 2 can be used and the calibration 
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object image photographed in accordance with an instruction for 
calibration photographing does not need to be used. Also, the 
user terminal 40 can obtain a corrected subject image from the 
subject image photographed by the camera 2, using the results 
5 of the calibration process obtained from the image calibration 
processing device 30. At this time, since the user terminal 
40 bears the burden of calculation processing for a corrected 
subject image, a corrected subject image can be reliably 
obtained even in cases where a communication line between the 

10 user terminal 4 0 and the image calibration processing device 
30 has a narrow bandwidth or is not stable. 

In the image calibration method above, preferably the 
calculated results of the calibration process are accumulated 
in the calibration database (19, 39), together with the model 

15 of camera, when a calibration process is performed using the 
calibration object image received from the user terminal (20, 
40) . 

In order to achieve the first object, an image calibration 
method according to the present invention comprises, as shown 

20 for example in FIG . 1 and FIG. 2, the steps of: transmitting 
a calibration request to an image calibration processing device 
10 from a user terminal 20 (S10) ; transmitting a calibration 
object image photographed by a predetermined camera 2 in 
accordance with an instruction for calibration photographing 

25 from the user terminal 20 to the image calibration processing 
device 10 (S16) ; transmitting a subject image photographed by 
the camera 2 used to photograph the calibration object image 
to the image calibration processing device 10 (S30) ; and 
receiving, at the user terminal, a corrected subject image 

30 transmitted from the image calibration processing device 10 and 
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formed by performing a calibration on the subject image in 
accordance with results of a calibration process obtained using 
the transmitted calibration object image (S34). 

In order to achieve the first object, an image calibration 
5 method according to the present invention, as shown for example 
in FIG. 14 and FIG. 15, comprises the steps of: transmitting 
a calibration request to an image calibration processing device 
30 from a user terminal 40 (S100) ; transmitting a calibration 
object image photographed by a predetermined camera 2 in 

10 accordance with an instruction for calibration photographing 
from the user terminal 40 to the image calibration processing 
device 30 (S106) ; receiving at the user terminal 40 results of 
a calibration process calculated by the image calibration 
processing device 30 (S118); and performing a calibration on 

15 a subject image photographed by the camera 2 used to photograph 
the calibration object image using the received results of the 
calibration process, to form a corrected subject image (S122) . 

In order to achieve the second object, an image calibration 
processing device 10 according to the present invention, 

20 comprises, as shown for example in FIG. 1, a receiving section 
for user terminal 11 for receiving a use request for image 
calibration transmitted from a user terminal 20, a calibration 
object image photographed in accordance with an instruction for 
calibration photographing, and a subject image photographed by 

25 a camera used to photograph the calibration object image; a 
calibration processing section 14 for performing a calibration 
process using the calibration object image received by the 
receiving section for user terminal 11; a corrected subject 
image forming section 15 for performing a calibration on the 

30 subject image in accordance with results of the calibration 
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process to form a corrected subject image; and a transmission 
section for user terminal 16 for returning the corrected subject 
image formed by the corrected subject image forming section 15 
to the user terminal 20. 
5 Preferably, the image calibration processing device 10 

according to the present invention further comprises: a display 
section 16a for displaying the corrected subject image formed 
by the corrected subject image forming section 15. Preferably , 
in the image calibration processing device 10 , as shown for 

10 example in FIG. 18, the calibration object image and the subject 
image are a common image in which a calibration object and a 
subject are photographed at a time, and the device comprises 
a three-dimensional measurement section 15a for performing a 
three-dimensional measurement on the subject using the 

15 corrected subject image. 

In order to achieve the second object, an image calibration 
processing device 30 according to the present invention 
comprises, as shown for example in FIG. 14, a receiving section 
for user terminal 31 for receiving a use request for image 

20 calibration transmitted from a user terminal 40 and a 
calibration object image photographed in accordance with an 
instruction for calibration photographing; a calibration 
processing section for performing a predetermined calibration 
process on the calibration object image received by the 

25 receiving section for user terminal 31; and a transmission 
section for user terminal 36 for returning results of the 
calibration process calculated by the calibration processing 
section 34 to the user terminal 40. And, the user terminal 40 
performs a calibration on a subject image photographed by a 

30 camera used to photograph the calibration object image using 
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the results of the calibration process transmitted from the 
transmission section for user terminal 36, to form a corrected 
sub j ect image . 

Preferably, in the image calibration processing device 30 
5 according to the present invention, as shown for example in FIG. 
20, the calibration object image and the subject image are a 
common image in which a calibration object 4 and a subject 6 
are photographed at a time, and the user terminal 40 is 
configured to further include a three-dimensional measurement 

10 section 47 performing a three-dimensional measurement on the 
subject using the corrected subject image. 

Preferably, the image calibration processing device (10, 
30) according to the present invention further comprises: an 
instruction retrieving section (13, 33) for retrieving the 

15 instruction suitable for the use request for image calibration 
received by the receiving section for user terminal (11, 31), 
and the transmission section for user terminal (16, 36) is 
configured to return the instruction retrieved by the 
instruction retrieving section (13, 33) to the user terminal 

20 (20, 40) . Preferably, the instruction described above further 
includes an image showing a photographing state. Such an image 
will facilitate understanding for a person in charge of 
photographing a calibration object image and a subject image. 
Preferably, in the image calibration processing device (10, 

25 30) according to the present invention, the receiving section 
for user terminal (11, 31) is configured to receive at least 
a focal length in the photographing and a model of the camera, 
or a focal length in the photographing and a size of an imaging 
device used in the photographing, together with the calibration 

30 object image or the subject image transmitted from the user 
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terminal (20, 40) . The focal length in the photographing and 
the model of the camera, or the focal length in the photographing 
and the size of an imaging device used in the photographing, 
are useful information for calibration processing. 
5 Preferably, the image calibration processing device (10, 30) 

according to the present invention includes an information 
reading section (14b, 34b) for reading data necessary for the 
calibration process from photographing information affixed to 
the calibration object image transmitted from the user terminal 

10 (20, 40) . The data read by the information reading section is 
used for the calibration process by the calibration processing 
section (14, 34). Preferably, in the image calibration 
processing device (10, 30) according to the present invention, 
the calibration processing section (14, 34) is configured to 

15 calculate results of the calibration process including at least 
one of a principal point position, distortion data, and a focal 
length of a lens used in the photographing, of the camera 
involved in the photographing of the subject image. 

In order to achieve the second object, an image calibration 

20 processing device 10 according to the present invention 
comprises, as shown for example in FIG. 12, a receiving section 
for user terminal 11 for receiving a calibration request and 
a subject image from a user terminal 20, together with a model 
of a camera 2 used to photograph the subject image; a calibration 

25 database 19 in which results of a calibration process are 
accumulated according to a model of a camera 2; a calibration 
process results retrieving section 14a for retrieving, from the 
calibration database 19, results of the calibration process 
corresponding to the received model of the camera; a corrected 

30 subject image forming section 15 for performing a calibration 
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on the subject image transmitted from the user terminal 20 using 
the retrieved results of the calibration process, to form a 
corrected subject image; and a transmission section for user 
terminal 16 for returning the formed corrected subject image 
5 to the user terminal 20. 

In order to achieve the second object, an image calibration 
processing device 30 comprises, as shown for example in FIG. 
16, a receiving section for user terminal 31 for receiving a 
calibration request from a user terminal 40, together with a 

10 model of a camera 2 used to photograph a subject image; a 
calibration database 39 in which results of a calibration 
process are accumulated according to a model of a camera; a 
calibration process results retrieving section 34a for 
retrieving, from the calibration database 39, results of the 

15 calibration process corresponding to the received model of the 
camera; and a transmission section for user terminal 36 for 
transmitting the retrieved results of the calibration process 
to the user terminal 40 . And, the user terminal 40 is configured 
to perform a calibration on the subject image using the results 

20 of the calibration process transmitted from the transmission 
section for user terminal 36, to form a corrected subject image. 

In order to achieve the third object, an image calibration 
processing terminal 20 comprises, as shown for example in FIG. 
1, a transmission section for processing device 21 for 

25 transmitting a use request for image calibration, a calibration 
object image photographed in accordance with an instruction for 
calibration photographing, and a subject image photographed by 
a camera 2 used to photograph the calibration object image, to 
an image calibration processing device 10; a receiving section 

30 for processing device 24 for receiving a corrected subject image 
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transmitted from the image calibration processing device 10 and 
formed by performing a calibration on the subject image; and 
a display section 25 for displaying the corrected subject image 
received by the receiving section for processing device 24. 
5 In order to achieve the third object, an image calibration 
processing terminal comprises, as shown for example in FIG. 14, 
a transmission section for processing device 41 for 
transmitting a use request for image calibration, and a 
calibration object image photographed in accordance with an 

10 instruction for calibration photographing, to an image 
calibration processing device 30; a receiving section for 
processing device 44 for receiving results of a calibration 
process transmitted from the image calibration processing 
device 30; and a corrected subject image forming section 4 6 for 

15 performing a calibration on a subject image photographed by a 
camera 2 used to photograph the calibration object image using 
the results of the calibration process received by the receiving 
section for processing device 44, to form a corrected subject 
image . 

20 

Effect of the Invention 

According to an image calibration method of the present 
invention, an image calibration processing device, located away 
from a user terminal, performs a calibration process on a camera, 

2,5 by performing the steps of receiving a calibration object image 
photographed in accordance with an instruction for calibration 
photographing with an image calibration processing device, and 
performing a calibration process using the calibration object 
image received. Since a calibration is performed on a subject 

30 image photographed by the camera used to photograph the 
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calibration object image in accordance with the results of the 
calibration process, to form a corrected subject image, a 
corrected subject image in which influences of the lens 
aberrations of the camera are compensated can be easily 
5 obtained. 

Also, an image calibration method of the present invention 
includes the steps of transmitting information on the model of 
a camera used to photograph a subject image from a user terminal 
to an image calibration processing device having a calibration 

10 database in which the results of a calibration process are 
stored according to the model of camera, and retrieving results 
of the calibration process corresponding to the model of the 
camera received from the calibration database. Therefore, if 
the results of the calibration process retrieved can be applied 

15 to a subject image from the camera, a user of the camera can 
easily obtain a corrected subject image without the need to 
photograph a calibration object image in accordance with an 
instruction for calibration photographing. 

The basic Japanese Patent Application No. 2002-272858 filed 

20 on September 19, 2002 is hereby incorporated in its entirety 
by reference into the present application . 

The present invention will become more fully understood from 
the detailed description given hereinbelow. The other 
applicable fields will become apparent with reference to the 

25 detailed description given hereinbelow. However, the detailed 
description and the specific embodiment are illustrated of 
desired embodiments of the present invention and are described 
only for the purpose of explanation. Various changes and 
modifications will be apparent to those ordinary skilled in the 

30 art on the basis of the detailed description. 
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The applicant has no intention to give to public any 
disclosed embodiments. Among the disclosed changes and 
modifications, those which may not fall within the scope of the 
present claims constitute, therefore, a part of the present 
5 invention in the sense of doctrine of equivalents. 

Brief Description of the Drawings 

FIG. 1 is a general block diagram illustrating the structure 
of a first embodiment of the present invention. 
10 FIG. 2 is a flowchart for illustrating a method of the first 

embodiment of the present invention. 

FIG. 3 is a plan view showing an example of a two-dimensional 
chart . 

FIG. 4 is a view illustrating an example of the results of 
15 a calibration process. 

FIG. 5 is a view illustrating an example of the results of 
the calibration process performed on a zoom lens. 

FIG. 6 is a flowchart showing the details of instructions used 
to photograph a calibration object image. 
20 FIG. 7 illustrates an example of an item entry sheet. 

FIG. 8 is a perspective view illustrating an arrangement of 
a camera when measuring the lens aberrations at various focal 
lengths of the camera. 

FIG. 9 illustrates a procedure for photographing the two- 
25 dimensional chart by means of the camera, where (Al) , (A2) , (A3) , 
and (A4) show an image from the camera, and (Bl) , (B2) , (B3) , 
and (B4) show positional relation between the two-dimensional 
chart and the camera 2 corresponding to the image from the 
camera . 

30 FIG. 10 is a diagram showing camera intervals when measuring 
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the lens aberrations at a focal length equivalent to a standard 
lens or a telephoto lens. 

FIG. 11 is a flowchart illustrating an example of a procedure 
for photographing a calibration object image in accordance with 
5 the instructions. 

FIG. 12 is a general block diagram illustrating the structure 
of a second embodiment of the present invention. 

FIG. 13 is a flowchart for illustrating a method of the second 
embodiment of the present invention. 
10 FIG. 14 is a general block diagram illustrating the structure 

of a third embodiment of the present invention. 

FIG. 15 is a flowchart for illustrating a method of the third 
embodiment of the present invention. 

FIG. 16 is a general block diagram illustrating the structure 
15 of a fourth embodiment of the present invention. 

FIG. 17 is a flowchart for illustrating a method of the fourth 
embodiment of the present invention. 

FIG. 18 is a general block diagram illustrating the structure 
of a fifth embodiment of the present invention. 
20 FIG. 19 is a flowchart for illustrating a method of the fifth 
embodiment of the present invention. 

FIG. 20 is a general block diagram illustrating the structure 
of a sixth embodiment of the present invention. 

FIG. 21 is a flowchart for illustrating a method of the sixth 
25 embodiment of the present invention. 

Best Mode for Carrying Out the Invention 
First Embodiment 

The present invention is described below with reference to 
30 the drawings. FIG. 1 is a general block diagram illustrating 
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the structure of a first embodiment of the present invention. 
As shown in the figure, an image calibration processing system 
of the present invention includes a camera 2, a calibration 
object 4, a subject 6, a communication line 8, an image 
5 calibration processing device 10, and a plurality of user 
terminals 20. The camera 2 is an optical instrument with 
relatively large lens aberrations such as, for example, a 
general-purpose single-lens reflex camera or digital camera, 
compared to cameras for high-precision measurement purposes. 

10 The calibration object 4 is a chart used to perform a calibration 
to compensate the lens aberrations, and may be a three- 
dimensional chart or a two-dimensional chart. A three- 
dimensional chart has reference points, of which respective 
two-dimensional positions and heights have been precisely found. 

15 A two-dimensional chart will be described in detail later. The 
subject 6 is an object to be photographed by the camera 2, and 
may be a subject for a panoramic photograph (for example a 
landscape) or a subject for two-dimensional or three- 
dimensional measurement (for example a work piece, a 

20 geographical feature, or the like) . The communication line 8 
may be a public line or a dedicated line, either wired or wireless, 
which allows data communication. 

The image calibration processing device 10 includes a 
receiving section for user terminal 11, a user authentication 

25 section 12, an instruction retrieving section 13, a calibration 
processing section 14, an information reading section 14b, a 
corrected subject image forming section 15, a transmission 
section for user terminal 16, a display section 16a, a user 
database 17, an instruction database 18, and a calibration 

30 database 19. An instrument with a high calculation capacity 
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such as, for example, a server or a workstation may be used as 
the image calibration processing device 10. 

The receiving section for user terminal 11 receives, via the 
communication line 8, a use request for image calibration, a 
5 calibration object image photographed in accordance with an 
instruction for calibration photographing, and a subject image 
of the subject 6 photographed by the camera 2 transmitted from 
a user terminal 20. The calibration object image and the 
subject image are basically images in which the calibration 

10 object 4 or the subject 6, respectively, is individually 
photographed by the common camera 2. The use request for image 
calibration may conveniently include photographing condition 
information useful for calibration processing, such as a model 
or focal length of the camera, and/or the like. The 

15 photographing condition information may be transmitted as 
independent data, or included as supplementary information in 
the calibration ob j ect image or the sub j ect image . For example , 
a basic format adopted for digital cameras called "Exif" 
includes, at the beginning of the format, photographing 

20 condition information such as a photographing instrument, date, 
aperture, shutter speed, and the like used in the photographing. 

The user authentication section 12 is used by an operator of 
the image calibration processing device 10 to certify a user 
of the user terminal 20 as having a proper title for use. The 

25 user authentication section 12 uses, for example, an address 
and name of the user, a number of a telephone line utilized as 
the communication line 8, an authentication key distributed by 
the image calibration processing device 10, and/or the like. 
The user database 17 has registered therein users of the user 

30 terminals 20 with a proper title for use. The user database 
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17 stores a registration date, a valid period of membership 
registration, and a number of times of use in cases where a 
membership system is adopted, or an accounting status in cases 
where a pay system is adopted. When the user authentication 
5 section 12 has certified a user of the user terminal 20 as having 
a proper title for use, the use request for image calibration 
received by the receiving section for user terminal 11 can then 
be dealt with. 

The instruction retrieving section 13 retrieves, from the 

10 instruction database 18, an instruction for calibration 
photographing suitable for the use request for image 
calibration received by the receiving section for user terminal 
11. When an instruction for calibration photographing is 
already available to the user of the user terminal 20, the 

15 instruction retrieving section 13 may not retrieve one. The, 
instruction database 18 stores instructions for calibration 
photographing according to the model of the camera 2 and/or the 
type of the calibration object 4. The models of the camera 2 
may include, for example, a zoom lens, a wide-angle lens, a 

20 telephoto lens, a panoramic lens, a standard lens, and/or the 
like. The types of the calibration object 4 may include, for 
example, a two-dimensional chart, a three-dimensional chart, 
and/or the like. 

The calibration processing section 14 performs a calibration 

25 process using the calibration object image received by the 
receiving section for user terminal 11. The calibration 
process is performed to find various parameters used to 
compensate the camera aberrations, and will be described in 
detail later. The information reading section 14b reads data 

30 necessary for the calibration process from the photographing 



24 



information affixed to the calibration object image transmitted 
from the user terminal 20. The information reading section 14b 
reads, for example, the photographing information included in 
the basic format for digital cameras "Exif." The calibration 
5 database 19 has accumulated therein results of the calibration 
process according to the model or the lens focal length of the 
camera 2 . 

The corrected subject image forming section 15 performs a 
calibration on the subject image received by the receiving 

10 section for user terminal 11 in accordance with the calibration 
process, to form a corrected subject image. The transmission 
section for user terminal 16 transmits, via the communication 
line 8, the instruction retrieved by the instruction retrieving 
section 13 and the corrected subject image formed by the 

15 corrected subject image forming section 15 to the user terminal 
20. The display section 16a may be a CRT or liquid crystal 
display screen. The display section 16a displays the corrected 
subject image formed by the corrected subject image forming 
section 15, the results of the calibration process by the 

20 calibration processing section 14 , the calibration ob j ect image 
and the subject image received by the receiving section for user 
terminal 11, and/or the like. 

The image calibration processing terminal 20 of the present 
invention includes a transmission section for processing device 

25 21, a key entry section 22, a receiving section for processing 
device 24, and a display section 25. A personal computer, a 
personal digital assistant, or a mobile phone terminal with a 
built-in camera, for example, may be used as the image 
calibration processing terminal 20. The transmission section 

30 for processing device 21 and the receiving section for 
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processing device 24 may be provided with a modem for connection 
to the communication line 8. The transmission section for 
processing device 21 transmits, via the communication line 8, 
a use request for image calibration, a calibration object image 
5 photographed in accordance with an instruction for calibration 
photographing, and a subject image photographed by the camera 
2 used to photograph the calibration object image, to the image 
calibration processing device 10. The use request for image 
calibration may be transmitted together with authentication 

10 information indicating that the user has a proper title for use, 
and/or the photographing condition information. 

The key entry section 22 is used when the user inputs 
alphanumeric, kana, and other operating commands, and may be 
formed with, for example, a keyboard, a touch panel utilizing 

15 the display section 25, a push button, and/or the like. The 
receiving section for processing device 24 receives the 
corrected subject image formed by performing a calibration on 
the subject image and transmitted from the image calibration 
processing device 10. The receiving section for processing 

20 device 24 also receives, via the communication line 8, an 
instruction for calibration photographing, when necessary. 
The display section 25 displays the instruction for calibration 
photographing and the corrected subject image received by the 
receiving section for processing device 24. The display 

25 section 25 may display a calibration object image or a subject 
image . 

The operation of an apparatus constituted as described above 
will be described. FIG. 2 is a flowchart for illustrating a 
. method of the first embodiment of the present invention. A user 
30 terminal 20 transmits a calibration request (service request) 



26 



for the camera 2 to the image calibration processing device 10 
via the communication line 8 (S10) . In response to the 
calibration request, the image calibration processing device 
10 authenticates whether or not a user of the user terminal 20 
5 has a proper title for use. When the user of the user terminal 
20 is certified as having a proper title for use as a result 
of authentication, the image calibration processing device 10 
returns an instruction for calibration photographing according 
to the model of the camera 2 to the user terminal 20 (S12) . The 

10 instruction for calibration photographing may be accompanied 
by, for example, a two-dimensional chart as the calibration 
object 4. In cases where the user terminal 20 has a three- 
dimensional chart as the calibration object 4, the three- 
dimensional chart may be used, instead of the two-dimensional 

15 chart transmitted, as the calibration object 4. 

On the user terminal 20 side, the calibration object 4 is 
photographed by the camera 2 in accordance with the instruction 
for calibration photographing (S14). The calibration object 
image photographed is transmitted from the user terminal 2 0 to 

20 the image calibration processing device 10 via the 
communication line 8, together with the photographing data 
(S16) . The image calibration processing device 10 makes a 
determination on the photographing quality of the calibration 
object image transmitted, for example, as to whether or not the 

25 image is unsuitable for the calibration process because the 
image is out of focus or the calibration object 4 image is too 
small (S18) . If the calibration object image is determined to 
be unsuitable, rephotographing of the calibration object 4 by 
the camera 2 is requested (S20) . At this time, it should be 

30 advantageous to give a comment suggesting to what should be paid 
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attention in order to obtain a calibration object image suitable 
for the calibration process. In response to the request for 
rephotographing, the user photographs the calibration object 
4 by the camera 2 (S22) , while paying attention to what is 
5 suggested in the comment. The calibration object image 
photographed is transmitted from the user terminal 20 to the 
image calibration processing device 10 via the communication 
line 8, together with the photographing data (S24), and a 
determination is made on the photographing quality of the 

10 calibration object image in S18. 

If the photographing quality of the calibration object image 
is suitable, the calibration processing section 14 performs a 
calibration process using the calibration object image received 
(S26) . The results of the calibration process are accumulated 

15 in the calibration database 19, together with the model of the 
camera. On the user terminal 20 side, the subject 6 is 
photographed by the camera 2 used to photograph the calibration 
object image (S28) . The user terminal 2 0 transmits the subject 
image to the image calibration processing device 10 via the 

20 communication line 8 (S30) . The image calibration processing 
device 10 performs a calibration on the subject image received, 
in accordance with the results of the calibration process 
obtained in S26, to form a corrected subject image (S32) . The 
corrected subject image formed is returned from the image 

25 calibration processing device 10 to the user terminal 20 (S34) . 

The image calibration processing device 10 may make a 
determination on the photographing quality of the subject image 
transmitted at this time. For example, a determination may be 
made as to whether or not the image is unsuitable for the 

30 correction process because the image is out of focus or the 



28 



subject 6 image is too small. If the subject image is determined 
to be unsuitable, rephotographing of the subject 6 by the camera 
2 is requested. In cases of a pay service, a process necessary 
for accounting is performed as use information is accumulated 
5 in the user database 17. 

The two-dimensional chart as the calibration object 4 is 
described. Here, the two-dimensional chart is exemplified by 
a sheet of paper or plastic on which predetermined marks are 
printed. However, the two-dimensional chart can be a flat 

10 screen on which an image with predetermined marks are arranged. 
FIG. 3 is a plan view showing an example of the two-dimensional 
chart. The two-dimensional chart is shaped as a flat sheet. 
Viewable first marks and second marks constituted of a plurality 
of dots are printed on the front side of the two-dimensional 

15 chart. The first marks are used in the calibration process, 
and also used to determine the photographing angle at which the 
camera 2 photographs the two-dimensional chart. The first 
marks are provided at least three locations on the two- 
dimensional chart, and preferably provided respectively in the 

20 four quadrants equally dividing the two-dimensional chart. 
The second marks, which may also be referred to as targets, are 
used to designate the position of the image data on the 
two-dimensional chart photographed by the camera 2 . 
Preferably, the second marks are distributed with a uniform 

25 density all over the two-dimensional chart. The second marks 
are provided preferably at 30 locations or more, and more 
preferably at about 100 - 200 locations, on the two-dimensional 
chart . 

In this example, the two-dimensional chart has a total of five 
30 first marks arranged thereon, which have a diamond external 
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shape and a mark at its center used commonly as the second mark. 
The first marks la, lb, lc, and Id are respectively provided 
in the four quadrants dividing the two-dimensional chart. The 
first mark la is located in the upper left quadrant, lb in the 
5 upper right quadrant, lc in the lower left quadrant, and Id in 
the lower right quadrant. The first mark le is provided at the 
origin common to all the quadrants. For example, the first 
marks la, lb, lc, and Id are positioned at an equal distance 
d from the first mark le. Here, an assumption is made that the 

10 two-dimensional chart is rectangular, the vertical interval 
between the first mark la, lb and the first mark le is h, and 
the horizontal interval between the first mark lc, Id and the 
first mark le is 1. On this assumption, the distance d between 
the first mark la, lb, lc, Id and the first mark le satisfies 

15 the following relation : 
d = (h 2 + 1 2 ) 1/2 (1) 

The first marks and the second marks should beforehand be 
printed so as to conform to desired dimensions, or subjected 
to a measurement of dimensions. The positions of the first 

20 marks and the second marks may conveniently be stored in the 
calibration database 19 beforehand, to allow them to be printed 
on a sheet so as to conform to the stored coordinates. 

FIG. 4 is a view illustrating an example of the results of 
the calibration process. The display section 16a may display, 

25 for example, a focal length, a principal point position, and 
distortion parameters as calibration elements. As for the 
distortion indicating the lens aberrations, a curve before 
correction 142, a curve after correction 144, and an ideally 
corrected curve 14 6 may be displayed in graphical form for easy 

30 understanding. 
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FIG. 5 shows an example of the results of the calibration 
process performed on a zoom lens, where (A) shows a coefficient 
kl, (B) a coefficient k2, (C) an x-axis deviation coefficient 
xO for an x-axis deviation between the camera principal point 
5 and the image center in an image coordinate system x, and (D) 
a y-axis deviation coefficient yO for a y-axis deviation between 
the camera principal point and the image center in an image 
coordinate system y. Here, the focal length of the zoom camera 
2 is assumed to be adjustable from 7.2 mm to 50.8 mm. With the 

10 number of measurement points for focal length data being six, 
7.2 mm and 52.2 mm are respectively used as the most wide-angle 
and most telephoto measurement points, and 8 . 7 mm, 11.6 mm, 24.8 
mm, and 39.0 mm are selected as intermediate measurement points 
uniformly distanced in terms of optics. The coefficients kl 

15 and k2 will be used in the equation (4) for a bundle adjusting 
method with self-calibration described later, and take maximum 
values on the most wide-angle side and smaller values on the 
most telephoto side. 

FIG. 6 is a flowchart showing the outline of the instructions 

20 used to photograph a calibration object image, illustrating the 
details at the calibration request (service request) stage. 
The image calibration processing device 10 transmits an item 
entry sheet to the user terminal 20 (S50) . The item entry sheet 
includes items necessary for the calibration process, such as 

25 whether a two-dimensional chart or an existing three- 
dimensional chart is used as a photographing target as the 
calibration object 4. 

The user terminal 20 prints or displays the item entry sheet 
transmitted (S52) , to allow entry or addition of the necessary 

30 items such as the camera 2, calibration object 4, subject 6, 
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and photographing conditions. Upon request from the user 
terminal 20, a two-dimensional chart as the calibration object 
4 is transmitted (S54) . The user terminal 20 prints or displays 
the two-dimensional chart as the calibration 4 transmitted 
5 (S56) . The user terminal 20 returns the item entry sheet after 
entering or adding the photographing conditions and/or the like 
to the image calibration processing device 10, to signal the 
completion of preparation for photographing (S58) . At this 
time, on the user terminal 20 side, the printed two-dimensional 

10 chart should be affixed on a wall, or the display section 25 
of the user terminal 20 should be placed in an appropriate 
position to display the two-dimensional chart transmitted. 
Depending on the camera angle as a photographing condition or 
the desired correction precision, the two-dimensional chart 

15 transmitted in S56 may not be satisfactory in terms of precision. 
In such a case, only a two-dimensional chart precisely printed 
or a three-dimensional chart precisely fabricated could attain 
the precision required for the corrected subject image, and 
therefore another two-dimensional chart or three-dimensional 

20 chart necessary has to be waited for to be mailed or transmitted. 

The image calibration processing device 10 transmits an 
instruction for calibration photographing corresponding to the 
model of the camera 2 to the user terminal 20 (S60) . The 
instruction may be transmitted as a batch file, or transmitted 

25 sequentially in real time as the photographing progresses like 
"streaming". On the user terminal 20 side an image of the 
calibration object 4 is photographed by the camera 2, in 
accordance with the instruction for calibration photographing 
(S62) . When the photographing of the calibration object has 

30 been completed, the user terminal 20 transmits a photographing 
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completion signal to the image calibration processing device 
10 (S64) . The image calibration processing device 10 prepares 
for the reception of the calibration object image, and then 
transmits an image transmission request signal to the user 
5 terminal 20 (S66) . 

FIG. 7 illustrates an example of the item entry sheet, showing 
a sheet screen to be transmitted online. The item entry sheet 
screen is provided with a default camera parameters 
configuration screen, a correction model configuration screen, 

10 and an other camera parameters confirmation screen. The 
default camera parameters configuration screen is provided with 
entry sections for a focal length default value, and an imaging 
device sizes and image resolutions. 

A description is made of a procedure for photographing the 

15 two-dimensional chart by means of the camera 2, as an 
instruction for calibration photographing. FIG. 8 is a 
perspective view illustrating an arrangement of a camera when 
measuring the lens aberrations at various focal lengths of the 
camera. Performing a calibration may be possible when two or 

20 more images of the two-dimensional chart photographed from 
different directions are available. In cases where a two- 
dimensional chart printed on a sheet is used, preferably the 
two-dimensional chart is photographed from three or more 
directions. In this manner, even for wide-angle lenses with 

25 a short focal length, measurement values for the respective 
calibration elements, in particular for the focal length, can 
be stable and reliable. 

FIG. 8 shows a procedure where the photographing is performed 
from five directions, namely the very front (I) , the upper left 

30 (II) , the upper right (III) , the lower left (IV) , and the lower 
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right (V) . With the depth precision at the actual photographing 
site set at about 1 cm, the photographing incident angle between 
the optical axis of the camera 2 and the two-dimensional chart 
is preferably in the range of 10 degrees - 30 degrees, and more 
5 preferably in the range of 12 degrees - 20 degrees, considering 
the in-focus distance associated with the focal depth of the 
lens being limited. Typically, the photographing incident 
angle between the optical axis of the camera 2 and the two- 
dimensional chart may conveniently be 15 degrees. The various 

10 focal lengths typically refer to focal lengths equivalent to 
a standard lens, a wide-angle lens, and a telephoto lens used 
for a single-lens reflex camera. 

With reference to FIG. 9, a description is made of a procedure 
for photographing the two-dimensional chart by means of the 

15 camera. In FIG. 9, (Al), (A2), (A3), and (A4) show an image 
from the camera, and (Bl) , (B2) , (B3) , and (B4) show positional 
relation between the two-dimensional chart and the camera 2 
corresponding to the image from the camera. The numbers (I) 
- (V) correspond to the camera positions shown in FIG. 8. 

20 (I) : The two-dimensional chart is photographed from the very 
front such that all the first marks and the second marks on the 
two-dimensional chart are distributed all over but still 
included in the resulting image (FIG. 9 (Al) and (Bl) ) . By 
allowing the first marks and the second marks to be distributed 

25 all over the photographed image to corners thereof, distortion 
correction can be securely performed to peripheral portions of 
the lens. The photographing distance H varies depending on the 
focal length of the camera. 

(II) : Then, in cases where the focal length of the zoom camera 
30 is equivalent to a telephoto lens or a standard lens, the camera 
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is moved to a position about one third of the photographing 
distance H away from the very front camera position. That is, 
the camera position is changed such that the first mark la in 
the upper left quadrant, for example, comes to the photographing 
5 center (FIG. 9 (A2) and (B2) ) . However, in cases where the focal 
length of the zoom camera is equivalent to a wide-angle lens 
and the photographing distance H is about 1 m or less, the camera 
2 can simply be moved such that the target first mark comes to 
the very front of the camera 2. While keeping the camera 2 in 

10 the same position, the camera 2 is directed such that the central 
first mark le comes to the center of the resulting image (FIG. 
9 (A3) and (B3) ) . The camera 2 is then moved closer to the 
two-dimensional chart such that the first marks and the second 
marks are distributed all over the image photographed by the 

15 camera 2 (FIG. 9 (A4) and (B4)). 

(III) : The camera position is changed such that the first mark 
lb in the upper right quadrant comes to the photographing center. 
While keeping the camera in the same position, the camera is 
directed. such that the central first mark le comes to the center 

20 of the resulting image, to perform photographing with the first 
marks and the second marks distributed all over the resulting 
image . 

(IV) : The camera position is changed such that the first mark 
lc in the lower left quadrant comes to the photographing center. 

25 While keeping the camera in the same position, the camera is 
directed such that the central first mark le comes to the center 
of the resulting image, to perform photographing with the first 
marks and the second marks distributed all over the resulting 
image . 

30 (V) : The camera position is changed such that the first mark 
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Id in the lower right quadrant comes to the photographing center. 
While keeping the camera in the same position, the camera is 
directed such that the central first mark le comes to the center 
of the resulting image, to perform photographing with the first 
5 marks and the second marks distributed all over the resulting 
image . 

By following the procedure as described above, the angle of 
the camera 2 can be secured as the difference between necessary 
photographing angles and therefore the focal length can be 

10 securely measured. 

The distance H between the camera 2 and the two-dimensional 
chart can be determined from the focal length f of the zoom camera . 
For example, in cases where the focal distance of the zoom camera 
is equivalent to a 35 mm standard lens, the photographing 

15 distance H is about 90 cm. Since the interval d between the 
first marks provided on the two-dimensional chart is, for 
example, 20 cm, an angle of about 10 degrees can be secured as 
the photographing angle, when the photographing direction is 
tilted from the very front (I) to the upper left (II) or the 

20 like. 

An upper limit of the tilt angle of the photographing direction 
is determined according to the focal depth or the like . In other 
words, when the tilt angle of the photographing direction is 
large, the distances between the camera 2 and the respective 

25 first marks differ greatly, resulting in blurred images of the 
first marks in the resulting image. Therefore, the upper limit 
of the tilt angle of the photographing direction is set, for 
example, to 30 degrees. The actual photographing procedure is 
as described above in (I) - (V) . Photographing such that the 

30 first marks and the second marks are distributed all over the 



36 



camera screen naturally satisfies the conditions above, and 
hence satisfies the conditions of the photographing distance 
and position. 

FIG. 10 is a diagram showing camera intervals when measuring 
5 the lens aberrations at a focal length equivalent to a standard 
lens or a telephoto lens. In cases where the focal length is 
equivalent to a standard lens or a telephoto lens, the narrow 
field angle of the photographing lens does not allow a large 
difference in the photographing angle. Therefore, an angle of 

10 10 degrees cannot be secured as the photographing angle, when 
the photographing direction is tilted from the very front (I) 
to the upper left (II) or the like. In other words, this is 
because the interval d between the first marks is only about 
20 cm while the photographing distance H between the camera 2 

15 and the two-dimensional chart is 1 m or more, in cases of a long 
focal length. To deal with this, using the very front camera 
position as the center, the left camera positions (II) and (IV) 
and the right camera positions (III) and (V) can be determined. 
At this time, the camera is positioned about one third of the 

20 photographing distance H away from the very front (I) position 
to the left or the right, to allow photographing from the upper 
left (II) and the lower left (IV), and the upper right (III) 
and the lower right (V) as described above. The optical axis 
of the camera may conform to the direction of a line normal to 

25 the two-dimensional chart, or may be directed toward the 
two-dimensional chart . 

In the instruction for calibration photographing described 
above, five photographing positions , namely the very front (I), 
the upper left (II) , the upper right (III) , the lower left (IV) , 

30 and the lower right (V) , are used. However, the number of 
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photographing positions may be two, namely left and right, at 
minimum, or may be three or more. In cases of the two, left 
and right, directions, the two-dimensional chart is 
photographed such that an angle of about 10 degrees can be 
5 secured as the photographing angle. 

Instructions for calibration photographing are described 
using a flowchart. FIG. 11 is a flowchart illustrating an 
example of a procedure for photographing a calibration object 
image in accordance with the instructions. The figure shows 

10 a case where the image calibration processing device 10 and the 
user terminal 20 transmit and receive the instructions in an 
interactive manner to photograph a calibration object image. 
However, the photographing procedures may all be transmitted 
to a client side at a time in a batch manner, to allow the client 

15 to perform photographing while referring to the instructions 
as appropriate on a computer. Here, a description will be made 
of a case where the two-dimensional chart is photographed from 
five directions, namely the very front (I), the upper left (II), 
the upper right (III) , the lower left (IV) , and the lower right 

20 (V), as shown in FIG. 8. 

With respect to the very front (I) direction, a photographing 
method 1 is transmitted from the image calibration processing 
device 10 to the user terminal 20 (S70) . The photographing 
method 1 allows photographing in a state shown, for example, 

25 in FIG. 9 (Al) and (Bl) . Preferably, the user terminal 20 
displays a simple message indicating the photographing 
procedure, saying, for example, "Set the camera 2 so that the 
central first mark le comes to the center and the entire 
two-dimensional chart fills the screen, and then photograph." 

30 The user terminal 20 may also display the diagrams of FIG. 9 
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(Al) and (Bl) , simultaneously with the message or sequentially 
upon request from the client. To display the first mark le as 
a photographing target on the display section 25, a switching 
operation is made to display a photographing target image, to 
5 allow photographing. Since photographing will be performed 
with the two-dimensional chart filling much of the screen, the 
photographing distance H can naturally be determined according 
to the field angle of the camera 2. 

The first mark le on the two-dimensional chart is photographed 

10 using the camera 2 (S72) . At this time, an operator photographs 
the calibration object 4 while checking if an image similar to 
that displayed according to the instruction is seen through a 
finder (monitor) . The image calibration processing device 10 
is notified of the completion of photographing the first mark 

15 le (S74) . If the instructions have been downloaded in a batch 
manner beforehand, a sequential communication at the completion 
of photographing the first mark le is not necessary. If the 
camera is connected to a computer or if the transmission of a 
photographed image is not a trouble, the photographed image may 

20 be transmitted in S74 . Immediate transmission of a calibration 
object image that has just been photographed will facilitate 
evaluation of the calibration object image and rephotographing 
of the calibration object in case of a low photographing 
quality. 

25 Photographing procedures for the four points on the two- 
dimensional chart in the respective quadrants, namely the first 
marks la, lb, lc, and Id, are sequentially displayed, to allow 
photographing of such points while checking, as follows. To 
photograph the first mark la, a procedure 1 of a photographing 

30 method 2 is transmitted (S76) . For example, FIG. 9 (A2) and 
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(B2), and a message are transmitted, which says, "Now, let's 
move on to photographing the first mark la. First of all, move 
the camera so that the center of the camera comes to the position 
of the first mark la, while keeping the photographing distance. 
5 The moving distance should be about one third of the 
photographing distance." In response, the user changes the 
photographing arrangement of the calibration object 4 and the 
camera 2 to an arrangement as shown in FIG. 9 (B2), and checks 
if an image seen through the finder is similar to that according 

10 to the message (S78) . A completion signal of the procedure 1 
is transmitted from the user terminal 20 to the image 
calibration processing device 10 (S80) . 

A procedure 2 is transmitted from the image calibration 
processing device 10 (S82 ) . As f or FIG. 9 (A3) and (B3) , a message 

15 is transmitted, which says, "Then, direct the camera so that 
the first mark le in the center position comes to the center, 
while keeping the camera in the same photographing position." 
In response, the user changes the photographing arrangement of 
the calibration object 4 and the camera 2 to an arrangement as 

20 shown in FIG. 9 (B3) , and checks if an image seen through the 
finder is similar to that according to the message (S84) . A 
procedure 2 completion signal is transmitted from the user 
terminal 20 to the image calibration processing device 10 (S86) . 
A procedure 3 is transmitted from the image calibration 

25 processing device 10 (S88) . As for FIG. 9 (A4) and (B4), a 
message is transmitted, which says, "Finally, move the camera 
forward so that the targets are filled in the entire screen, 
while keeping the first mark le at the center. And press the 
shutter button." In response, the user changes the 

30 photographing arrangement of the calibration object 4 and the 
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camera 2 to an arrangement as shown in FIG. 9 (B4) , and checks 
if an image seen through the finder is similar to that according 
to the message (S90) . A photographing completion signal for 
the first mark le is transmitted from the user terminal 20 to 
5 the image calibration processing device 10 (S92) . The 
resulting image may be transmitted in S92, in which case the 
image can be evaluated to allow retransmitting of a 
photographing method. In such a case, procedures similar to 
those in S18, S20, S22, and S24 of FIG. 2 are taken. 

10 The other three first marks lb, lc, and Id on the two- 
dimensional chart are respectively subjected to procedures 1 
- 3 of photographing methods 3-5, which are similar to the 
procedures 1 - 3 of the photographing method 2 for the first 
mark la, in procedures similar to those in S76 - S92 of FIG. 

15 11. 

Second Embodiment 

FIG. 12 is a general block diagram illustrating the structure 
of a second embodiment of the present invention. In FIG. 12, 

20 components having the same functions as those in FIG. 1 
described above are given the same reference numerals and 
symbols, and descriptions of such components will not be 
repeated. The image calibration processing device 10 includes 
a calibration process results retrieving section 14a, in 

25 addition to the calibration processing section 14 and the 
calibration database 19. The calibration process results 
retrieving section 14a retrieves, from the calibration database 
19, results of the calibration process corresponding to the 
model of the camera received, utilizing the results of the 

30 calibration process accumulated in the calibration database 19 
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according to the camera model data. In cases where normal 
values are required in terms of precision for the corrected 
subject image, the results of the calibration process 
accumulated in the calibration database 19 can be utilized to 
5 substitute for the calibration process performed by the 
calibration processing section 14 on the specific camera 2 . 

The operation of an apparatus constituted as described above 
will be described. FIG. 13 is a flowchart for illustrating a 
method of the second embodiment of the present invention. A 

10 user terminal 20 transmits a calibration request (service 
request) for the camera 2 to the image calibration processing 
device 10 via the communication line 8 (S10) . When the user 
of the user terminal 20 is certified as having a proper title 
for use as a result of authentication, the image calibration 

15 processing device 10 requests model data on the camera 2 (Slla) . 
The user terminal 20 side acquires the model data on the camera 
2, by allowing entry using the key entry section 22, using an 
identification function for the camera 2 provided in the user 
terminal 20, or the like, to transmit the acquired model data 

20 to the image calibration processing device 10 via the 
communication line 8 (Sllb) . The calibration process results 
retrieving section 14a searches the results of the calibration 
process stored in the calibration database 19, to retrieve the 
corresponding model data on the camera (Sllc) . 

25 If the calibration process results retrieving section 14a has 
been able to retrieve the corresponding model data on the camera, 
the process proceeds to S28. At this time, it may be 
advantageous to consider variations in the results of the 
calibration process for the model data on the camera retrieved 

30 from the calibration database 19. For example, in cases where 



42 



a high precision is required in the corrected subject image, 
the process may be configured to proceed to S28 only when the 
obtained results of the calibration process satisfy the 
required precision. The descriptions of S28 - S34 have already 
5 been made in relation to FIG. 2, and therefore are not repeated 
here . 

On the other hand, if the calibration process results 
retrieving section 14a has been unable to retrieve the 
corresponding model data on the camera in Slid, the process 

10 proceeds to S12, where a calibration object image is 
photographed using the camera 2 to acquire results of the 
calibration process suitable for the specific camera 2. The 
descriptions of S12 - S26 have already been made in relation 
to FIG. 2, and therefore are not repeated here. 

15 According to the second embodiment of the present invention, 
the results of the calibration process are accumulated in the 
calibration database 19, and results of the calibration process 
corresponding to the model data on the camera are retrieved and 
transmitted to the user terminal, upon request from the user. 

20 Thus, the second embodiment of the present invention provides 
a significantly convenient method that allows the user to 
acquire intended results of the calibration process, even 
without the need to photograph a calibration object image. In 
the image calibration processing device 10, by accumulating 

25 results of the calibration process in the calibration database 
19, the model data on cameras to be supported can be enhanced 
due to the cumulative effect, thereby improving the efficiency. 

Third Embodiment 

30 FIG. 14 is a general block diagram illustrating the structure 
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of a third embodiment of the present invention. The third 
embodiment is different , compared with the first embodiment, 
in that the corrected subject image forming section is provided 
on the user terminal 4 0 side. In other words , the image 
5 calibration processing device 30 is configured that the results 
of the calibration process calculated by the calibration 
processing section 34 are transmitted from the transmission 
section for user terminal 36 to the user terminal 40. The user 
terminal 40 is provided with a corrected subject image forming 

10 section 4 6 for processing a subject image into a corrected 
subject image. For descriptions of components of the third 
embodiment shown in FIG. 14 , the descriptions of those of the 
first embodiment shown in FIG. 1 with the same names should be 
referred to, as long as the natures of the two counterparts will 

15 not contradict, and the detailed description will not be 
repeated. 

As shown in the figure, the image calibration processing 
device 30 includes a receiving section for user terminal 31, 
a user authentication section 32, an instruction retrieving 

20 section 33, a calibration processing section 34, an information 
reading section 34b, a transmission section for user terminal 
36, a display section 36a, a user database 37, an instruction 
database 38, and a calibration database 39. The image 
calibration processing terminal 40 includes a transmission 

25 section for processing device 41, a key entry section 42, a 
receiving section for processing device 44, a display section 
45, and a corrected subject image forming section 46. 

The operation of an apparatus constituted as described above 
will be described. FIG. 15 is a flowchart for illustrating a 

30 method of the third embodiment of the present invention. A user 
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terminal 40 transmits a calibration request (service request) 
for the camera 2 to the image calibration processing device 30 
via the communication line 8 (S100) . In response to the 
calibration request, the image calibration processing device 
5 30 authenticates whether or not a user of the user terminal 40 
has a proper title for use. When the user of the user terminal 
40 is certified as having a proper title for use as a result 
of authentication, the image calibration processing device 30 
returns an instruction for calibration photographing according 

10 to the model of the camera 2 to the user terminal 40 (S102) . 

On the user terminal 40 side the calibration object 4 is 
photographed by the camera 2, in accordance with the instruction 
for calibration photographing (S104) . The calibration object 
image photographed is transmitted from the user terminal 4 0 to 

15 the image calibration processing device 30 via the 
communication line 8, together with the photographing data 
(S106) . The image calibration processing device 30 makes a 
determination on the photographing quality of the calibration 
object image transmitted, for example, as to whether or not the 

20 image is unsuitable for the calibration process because the 
image is out of focus or the calibration object 4 image is too 
small (S108) . If the calibration object image is determined 
to be unsuitable, rephotographing of the calibration object 4 
by the camera 2 is requested (S110) . In response to the request 

25 for rephotographing, the user photographs the calibration 
object 4 by the camera 2 (S112) , while paying attention to what 
is suggested in the comment. The calibration object image 
photographed is transmitted from the user terminal 4 0 to the 
image calibration processing device 30 via the communication 

30 line 8, together with the photographing data (S114), and a 
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determination is made on the photographing quality of the 
calibration object image in S108. 

If the photographing quality of the calibration object image 
is suitable, the calibration processing section 14 performs a 
5 calibration process using the calibration object image received 
(S116) . The results of the calibration process are accumulated 
in the calibration database 19, together with the model of the 
camera- The model of the camera is acquired, for example, by 
reading the photographing state information attached to the 
10 calibration object image in the information reading section 14b. 
The image calibration processing device 30 returns the results 
of the calibration process obtained in S116 to the user terminal 
40 (S118) . 

On the user terminal 40 side the subject 6 is photographed 
15 by the camera 2 used to photograph the calibration object image 
(S120) . The user terminal 40 performs a calibration on the 
subject image in accordance with the results of the calibration 
process receiving the subject image, to form a corrected subject 
image (S122) . The corrected subject image formed may be, for 
20 example, displayed on the display section 45, printed on a paper 
medium, or accumulated in an electromagnetic storage medium. 

Fourth Embodiment 

FIG. 16 is a general block diagram illustrating the structure 

25 of a fourth embodiment of the present invention. In FIG . 16, 
components having the same functions as those in FIG. 1, FIG. 
12, and FIG. 14 described above are given the same reference 
numerals and symbols, and descriptions of such components will 
not be repeated. The image calibration processing device 30 

30 includes a calibration process results retrieving section 34a, 
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in addition to the calibration processing section 34 and the 
calibration database 39. The corrected subject image forming 
section 46 is provided on the user terminal 40 side, and not 
in the image calibration processing device 30. The calibration 
5 process results retrieving section 34a retrieves, from the 
calibration database 39, results of the calibration process 
corresponding to the model of camera received, utilizing the 
results of the calibration process accumulated in the 
calibration database 39 according to the camera model data. 

10 The operation of an apparatus constituted as described above 
will be described. FIG. 17 is a flowchart for illustrating a 
method of the fourth embodiment of the present invention. A 
user terminal 40 transmits a calibration request (service 
request) for the camera 2 to the image calibration processing 

15 device 30 via the communication line 8 (S100) . When a user of 
the user terminal 40 is certified as having a proper title for 
use as a result of authentication, the image calibration 
processing device 30 requests model data on the camera 2 (SlOla) . 
The user terminal 4 0 side acquires the model data on the camera 

20 2, by allowing entry using the key entry section 22, using an 
identification function for the camera 2 provided in the user 
terminal 40, or the like, to transmit the acquired model data 
to the image calibration processing device 30 via the 
communication line 8 (SlOlb) . The calibration process results 

25 retrieving section 34a searches the results of the calibration 
process stored in the calibration database 39, to retrieve the 
corresponding model data on the camera (SlOlc) . 

If the calibration process results retrieving section 34a has 
been able to retrieve the corresponding model data on the camera, 

30 the process proceeds to S118. The descriptions of S118 - S122 
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have already been made in relation to FIG. 15, and therefore 
are not repeated here. 

On the other hand, if the calibration process results 
retrieving section 34a has been unable to retrieve the 
5 corresponding model data on the camera in SlOld, the process 
proceeds to S112, where a calibration object image is 
photographed using the camera 2 to acquire results of the 
calibration process suitable for the specific camera 2. The 
descriptions of S112 - S116 have already been made in relation 
10 to FIG. 15, and therefore are not repeated here. 

Fifth Embodiment 

FIG. 18 is a general block diagram illustrating the structure 
of a fifth embodiment of the present invention. In FIG. 18, 

15 components having the same functions as those in FIG. 1 
described above are given the same reference numerals and 
symbols, and descriptions of such components will not be 
repeated. In this embodiment, the camera 2 photographs the 
calibration object 4 and the subject 6 as one image, and the 

20 user terminal 20 deals with a resulting stereo image 
photographed from at least two, namely left and right, 
photographing directions. The image calibration processing 
device 10 is provided with a three-dimensional measurement 
section 15a between the corrected subject image forming section 

25 15 and the transmission section for user terminal 16. 

The three-dimensional measurement section 15a includes a 
stereo image generating section 151 and an image measurement 
section 152. The stereo image generating section 151 has a 
function of generating a rectified image by performing an 

30 orientation (relative orientation and absolute orientation) 
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and a rectification on the stereo image photographed by the 
camera 2. Typically, the stereo image generating section 151 
comprises a computer-executable software program. Here, the 
rectified image refers to a pair of images photographed in 
5 stereo from the right and left photographing directions and 
subjected to a rectification and an adjustment so as to be viewed 
stereoscopically. The image measurement section 152 performs 
a so-called absolute orientation on the stereo image generated 
by the stereo image generating section 151, to calculate ground 

10 coordinates of each pixel. The details of this processing are 
disclosed, for example, in JP-A-Hei 11-351865. The use of the 
image measurement section 152 allows accurate measurement of 
a three-dimensional shape such as projections and depressions 
on the measuring object 30 surface. 

15 The operation of an apparatus constituted as described above 
will be described. FIG. 19 is a flowchart for illustrating a 
method of the fifth embodiment of the present invention. A user 
terminal 20 transmits a calibration request (service request) 
for the camera 2 to the image calibration processing device 10 

20 via the communication line 8 (S200) . The calibration request 
includes a request for a three-dimensional image measurement 
on the subject 6 using a stereo image. In response, the image 
calibration processing device 10 authenticates whether or not 
a user of the user terminal 20 has a proper title for use. When 

25 the user of the user terminal 20 is certified as having a proper 
title for use as a result of authentication, the image 
calibration processing device 10 returns an instruction for 
calibration photographing according to the model of the camera 
2 to the user terminal 20 (S202) . The instruction for 

30 calibration photographing includes important notices in a 
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three-dimensional image measurement on the subject 6 using a 
stereo image . 

On the user terminal 20 side the calibration object 4 and the 
subject 6 is photographed in stereo at a time so as to be included 
5 in one image, by the camera 2, in accordance with the instruction 
for calibration photographing (S204) . This photographing in 
stereo yields a pair of images photographed in stereo from the 
right and left photographing directions using the camera 2 . The 
calibration obj ect /sub j ect image photographed in stereo is 

10 transmitted from the user terminal 20 to the image calibration 
processing device 10 via the communication line 8, together with 
the photographing data (S206) . 

The image calibration processing device 10 makes a 
determination on the photographing quality of the calibration 

15 object/subject image transmitted, for example, as to whether 
or not the image is unsuitable for the calibration process 
because the image is out of focus or the calibration object 4 
image is too small (S208) . If the calibration obj ect /sub j ect 
image is determined to be unsuitable, rephotographing of the 

20 calibration object 4 and the subject 6 by the camera 2 is 
requested (S210) . At this time, it should be advantageous to 
give a comment suggesting to what should be paid attention in 
order to obtain a calibration object/subject image suitable for 
the calibration process. In response to the request for 

25 rephotographing, the user photographs the calibration object 
4 and the subject 6 by the camera 2 (S212) , while paying attention 
to what is suggested in the comment. The calibration 
obj ect /subject image photographed is transmitted from the user 
terminal 20 to the image calibration processing device 10 via 

30 the communication line 8, together with the photographing data 
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(S214) , and a determination is made on the photographing quality 
of the calibration ob j ect/sub j ect image in S208. 

If the photographing quality of the calibration 
object /subject image is suitable, the calibration processing 
5 section 14 performs a calibration process using the calibration 
object 4 image included in the calibration obj ect/sub j ect image 
received (S216) . The results of the calibration process are 
accumulated in the calibration database 19, together with the 
model of the camera. On the user terminal 20 side a corrected 

10 subject image is generated from the calibration object/subject 
image (S218), and is transmitted to the user terminal 20 via 
the communication line 8 (S220). In addition, in the image 
calibration processing device 10, the three-dimensional 
measurement section 15a performs a three-dimensional 

15 measurement on the corrected subject image. The three- 
dimensional measurement results obtained are returned from the 
image calibration processing device 10 to the user terminal 20 
(S224). At this time, the corrected subject image may be 
converted into an ortho image and then returned from the image 

20 calibration processing device 10 to the user terminal 20. The 
foregoing description is directed to a case where a three- 
dimensional measurement is performed by a stereo method using 
two images. However, various methods may alternatively be 
adopted, such as a single-photograph orientation method using 

25 a single image in which the calibration object 4 and the subject 
6 are photographed, or a bundle adjusting method with self- 
calibration using a plurality of images. 

Sixth Embodiment 

30 FIG. 20 is a general block diagram illustrating the structure 
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of a sixth embodiment of the present invention. In FIG. 20, 
components having the same functions as those in FIG. 18 
described above are given the same reference numerals and 
symbols, and descriptions of such components will not be 
5 repeated. In this embodiment, the three-dimensional 

measurement section is provided on the user terminal 40 side, 
and not in the image calibration processing section 30. The 
three-dimensional measurement section 47 includes a stereo 
image generating section 471 and an image measurement section 

10 472. For descriptions of the stereo image generating section 
471 and the image measurement section 472, the descriptions of 
the stereo image generating section 151 and the image 
measurement section 152 should be referred to. 

The operation of an apparatus constituted as described above 

15 will be described. FIG. 21 is a flowchart for illustrating a 
method of the sixth embodiment of the present invention. 
Descriptions of S200 - S216 of FIG. 21 are the same as those 
of S200 - S216 of FIG. 19, and therefore will not be repeated. 
The results of the calibration process in S216 are accumulated 

20 in the calibration database 19, together with the model of the 
camera, and transmitted to the user terminal 40 (S230) . The 
user terminal 40 uses the results of the calibration process 
transmitted, to generate a corrected subject image from the 
calibration ob j ect /sub j ect image (S232) . In addition, in the 

25 user terminal 40, the three-dimensional measurement section 47 
performs a three-dimensional measurement on the corrected 
subject image (S234) . At this time, the corrected subject image 
may be converted into an ortho image. The corrected subject 
image may be, for example, displayed on the display section 45 

30 of the user terminal 40, displayed on a television set for stereo 
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display, or the like. 

The bundle adjusting method with self -calibration used in the 
first through sixth embodiments is described. The calibration 
processing performed by the calibration processing section 14, 
5 34 uses a "bundle adjusting method with self -calibration" used 
in the field of photogrammetry for example. The term "bundle 
adjusting" refers to a method of adjusting the position and the 
tilt of the camera (exterior orientation elements) and the 
coordinate positions of the second marks at the same time, by 

10 establishing observation equations for every ray bundle of each 
image based on a collinearity condition that a ray bundle 
connecting a subject, a lens, and a CCD surf ace must be collinear, 
and applying a least squares method. The term "with self- 
calibration" indicates that the calibration elements, namely 

15 interior localizations of the camera (lens aberrations, 
principal point, and focal length) can also be found with the 
method. The basic equations of the collinear condition for the 
bundle adjusting method with self-calibration (hereinafter 
referred to as "bundle adjusting method" for simplicity) are 

20 represented by (Equation 2) and (Equation 3) as follows: 

x _ c a n (X-X 0 ) + a l2 (Y-Y 0 ) + a n (Z-Z 0 ) .... 
a 3x {X - X 0 ) + a 32 (Y-Y 0 ) + a 33 (Z-Z 0 ) 



.._ c S 2l (X-X 0 ) + a22 (Y-Y 0 ) + a 23 (Z-Z 0 ) .... 
a 3l (X - X 0 ) + a 32 (F - Y 0 ) + a 33 (Z - Z 0 ) 

25 

Here, the elements a xl - a 33 are elements of a rotation matrix 
at a camera tilt (co, cp, k) . The symbols co, cp, k represent the 
tilts of the camera at the time of photographing with respect 
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to the three axes X, Y, Z constituting the object coordinate 
system, and are called exterior orientation elements. (X, Y, 
Z) represents coordinates of an object such as a reference mark 
in the object coordinate system, and (X 0 , Y 0 , Z 0 ) represents 
5 coordinates of a projection center point Oc. (See the 
specification of Japanese Patent Application No. 2002-195058, 
paragraphs [0052], [0056], and [0068].) The bundle adjusting 
method is used to perform a least squares approximation using 
(Equation 2) and (Equation 3) with a plurality of images to find 

10 various solutions. The bundle adjusting method allows 
calculating the exterior orientation elements of the camera at 
respective photographing positions at the same time. In other 
words, the method allows calculation of the calibration 
elements of the camera. 

15 As an example of a correction model for the interior 
localizations (lens aberrations) , radial lens distortions can 
be represented by (Equation 4) as follows: 



Ax = x 0 + x(k x r 2 + k 2 r A ) 
Ay = y 0 + y(k[r 2 + k 2 r A ) (4) 

L r 2 =(x 2 +y 2 )/c 2 



20 k lf k 2 : radial lens distortion 



The correction model is not limited to this example, but may 
be selected as suitable for the lens used. Such calculations 
can be made with a successive approximation method, if six or 
25 more reference points are available in the ground coordinate 
system and the image coordinate system. 

Calculation of lens aberrations using a two-dimensional chart 
shown in FIG. 3 as the calibration object 4 is already disclosed 
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in the specification of Japanese Patent Application No. 
2002-195058 proposed by the present applicant, or the like . The 
specifics of the calculation of lens aberration using a 
two-dimensional chart are hardly related to the subject of the 
5 present invention, and therefore the specification is 
incorporated herein by reference in order to avoid redundancy. 
As an alternative to the two-dimensional chart as shown in FIG. 
3, other types of two-dimensional chart or three-dimensional 
chart may be used as the calibration object 4. The calibration 
10 processing can be performed using the obtained equations for 
calculating lens aberrations corresponding to the respective 
charts. 



